A universal, operational theory of unicast multi-user communication with fidelity criteria

This is a three part paper. Optimality of source-channel separation for communication with a fidelity criterion when the channel is compound as defined in [1] and general as defined in [2] is proved in Part I. It is assumed that random codes are permitted. The word “universal” in the title of this paper refers to the fact that the channel model is compound. The proof uses a layered black-box or a layered inputoutput view-point. In particular, only the end-to-end description of the channel as being capable of communicating a source to within a certain distortion level is used when proving separation. This implies that the channel model does not play any role for separation to hold as long as there is a source model. Further implications of the layered black-box view-point are discussed. Optimality of source-medium separation for multi-user communication with fidelity criteria over a general, compound medium in the unicast setting is proved in Part II, thus generalizing Part I to the unicast, multi-user setting. Part III gets to an understanding of the question, “Why is a channel which is capable of communicating a source to within a certain distortion level, also capable of communicating bits at any rate less than the infimum of the rates needed to code the source to within the distortion level”: this lies at the heart of why optimality of separation for communication with a fidelity criterion holds. The perspective taken to get to this understanding is a randomized covering-packing perspective, and the proof is operational. I. PART I: POINT-TO-POINT SETTING A. Introduction to, and contribution of Part I Optimality of separation based architectures for communication with a fidelity criterion over a discrete memoryless channel is proved in [3]. Optimality refers to the fact that if communication of some source to within some distortion level can be accomplished with some architecture, communication of the same source to within the same distortion level can also be accomplished with a source-channel separation based architecture. This result can be generalized to indecomposable channels, that is, finite state Markoff channels for which the state information dies down with time, as defined rigorously in [4]. Part I generalizes this optimality to the case when the channel is compound, that is, the channel belongs to a set, as defined in [1], and the channel transition probability is general, as defined in [2], with the difference that the probability of excess distortion criterion which is essentially the same as the criterion in [1] is used as the fidelity criterion instead of the expected distortion criterion used in [3]. The use of the word “universal” in the title of this paper refers to the fact that the result holds for a compound channel. Note that the universality is over the channel, not the source. A generalization to the compound setting is needed because the action of real media like wireless medium or the internet cannot be modeled as a known transition probability and one way to model these media might be that they belong to a set of transition probabilities. The multi-user generalization of Part I in the unicast setting is the subject of Part II. In order to prove this optimality, encoders and decoders are allowed to be random. That is, the encoder is a probability distribution on the set of deterministic encoders, the decoder has access to the particular realization of the encoder, and based on this access, acts as a probability distribution on the set of deterministic decoders. Error is calculated by averaging over the random code. This is called random coding. Randomcoding argument in [5] uses random codes. The difference is that in [5], random-coding is a proof technique whereas in the argument in Part I, random-coding is necessary in the sense that separation is not optimal for communication with a fidelity criterion over a general, compound channel if random codes are not permitted. The proof uses a layered black-box or a layered input-output view-point, and the proof style is different from that used in [3]. The question arises: can a proof in the style similar to [3] be used to prove the optimality of separation for communication with a fidelity criterion when the channel is compound. The answer is yes: Amos Lapidoth showed the first author, how to prove the result using techniques similar to [3] in a private discussion when the first author was explaining the result to Amos Lapidoth. A further question arises: what is the need for a different proof technique? The answer is many fold: First, the proof technique in [3] and the further extension due to Amos Lapidoth require an indecomposability assumption on the channel whereas the proof technique in Part I works for general channels as defined in [2]. It is for this reason that the probability of excess distortion criterion is used instead of the expected distortion criterion. The use of the probability of excess distortion criterion instead of the expected distortion criterion is similar in spirit as the use of the inf information rate in [2] instead of mutual information: in [2], a formula for channel capacity is given in terms of the inf information rate which is the lim inf in probability (see [2]) of the normalized information densities where the information density is iXnWn(a ; b) , log PY n|Xn(b |a) PY n(bn) (1) as compared to the usual formula for channel capacity which ar X iv :1 30 2. 58 60 v1 [ cs .I T ] 2 4 Fe b 20 13